{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_3xxr5fb",
    "id": "B116BFDF0D464FF49A85A582357D0B4D",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "<font color=red > 准备工作: 在本机上安装anaconda,然后根据https://pytorch.org/get-started/locally/ 安装torch环境,文件较大,耗时较久.除本节外,后边的代码均在和鲸的平台,利用d2l-pytorch环境运行.\n",
    "笔记以课程对应的notebook为基础,根据对相关书籍和辅修课程的学习补充笔记到notebook中.\n",
    "</font> \n",
    "\n",
    "# 线性回归\n",
    "主要内容包括：\n",
    "\n",
    "1. 线性回归的基本要素\n",
    "2. 线性回归模型从零开始的实现\n",
    "3. 线性回归模型使用pytorch的简洁实现"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_ht8ukap",
    "id": "8FCA1BC77B7F479BA1398473C2691BB0",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 线性回归的基本要素\n",
    "\n",
    "### 模型\n",
    "为了简单起见，这里我们假设价格只取决于房屋状况的两个因素，即面积（平方米）和房龄（年）。接下来我们希望探索价格与这两个因素的具体关系。线性回归假设输出与各个输入之间是线性关系:\n",
    "\n",
    "\n",
    "$$\n",
    "\\mathrm{price} = w_{\\mathrm{area}} \\cdot \\mathrm{area} + w_{\\mathrm{age}} \\cdot \\mathrm{age} + b\n",
    "$$\n",
    "\n",
    "\n",
    "\n",
    "### 数据集\n",
    "我们通常收集一系列的真实数据，例如多栋房屋的真实售出价格和它们对应的面积和房龄。我们希望在这个数据上面寻找模型参数来使模型的预测价格与真实价格的误差最小。在机器学习术语里，该数据集被称为训练数据集（training data set）或训练集（training set），一栋房屋被称为一个样本（sample），其真实售出价格叫作标签（label），用来预测标签的两个因素叫作特征（feature）。特征用来表征样本的特点。\n",
    "### 损失函数\n",
    "在模型训练中，我们需要衡量价格预测值与真实值之间的误差。通常我们会选取一个非负数作为误差，且数值越小表示误差越小。一个常用的选择是平方损失函数(squared loss,对预测误差大的有更大的惩罚,容忍很小的预测误差,可以有效地提升模型的泛化能力)。 它在评估索引为 $i$ 的样本误差的表达式为\n",
    "\n",
    "\n",
    "$$\n",
    "l^{(i)}(\\mathbf{w}, b) = \\frac{1}{2} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2,\n",
    "$$\n",
    "\n",
    "每次取一个小批量,就是下边的计算公式\n",
    "\n",
    "$$\n",
    "L(\\mathbf{w}, b) =\\frac{1}{n}\\sum_{i=1}^n l^{(i)}(\\mathbf{w}, b) =\\frac{1}{n} \\sum_{i=1}^n \\frac{1}{2}\\left(\\mathbf{w}^\\top \\mathbf{x}^{(i)} + b - y^{(i)}\\right)^2.\n",
    "$$\n",
    "\n",
    "\n",
    "### 优化函数 - 随机梯度下降\n",
    "当模型和损失函数形式较为简单时，上面的误差最小化问题的解可以直接用公式表达出来。这类解叫作解析解（analytical solution）。本节使用的线性回归和平方误差刚好属于这个范畴。然而，大多数深度学习模型并没有解析解，只能通过优化算法有限次迭代模型参数来尽可能降低损失函数的值。这类解叫作数值解（numerical solution）。\n",
    "\n",
    "在求数值解的优化算法中，小批量随机梯度下降（mini-batch stochastic gradient descent,综合了批量梯度下降BGD和随机梯度下降SGD的优点）在深度学习中被广泛使用。它的算法很简单：先选取一组模型参数的初始值，如随机选取；接下来对参数进行多次迭代，使每次迭代都可能降低损失函数的值。在每次迭代中，先随机均匀采样一个由固定数目训练数据样本所组成的小批量（mini-batch）$\\mathcal{B}$，然后求小批量中数据样本的平均损失有关模型参数的导数（梯度），最后用此结果与预先设定的一个正数的乘积作为模型参数在本次迭代的减小量。   \n",
    "\n",
    "$$\n",
    "(\\mathbf{w},b) \\leftarrow (\\mathbf{w},b) - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_{(\\mathbf{w},b)} l^{(i)}(\\mathbf{w},b)\n",
    "$$\n",
    "  \n",
    "学习率: $\\eta$代表在每次优化中，能够学习的步长的大小,   \n",
    "批量大小: $\\mathcal{B}$是小批量计算中的批量大小batch size   \n",
    "\n",
    "学习率和批量大小是人为设定的,不是通过模型训练学出的,因此称之为超参数(hyperparameter).\n",
    "\n",
    "总结一下，优化函数的有以下两个步骤：\n",
    "\n",
    "- (i)初始化模型参数，一般来说使用随机初始化；\n",
    "- (ii)我们在数据上迭代多次，通过在负梯度方向移动参数来更新每个参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_v3gyr0b",
    "id": "469D697FF90B48B7B0B61AED429EB8D6",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 矢量计算\n",
    "在模型训练或预测时，我们常常会同时处理多个数据样本并用到矢量计算。在介绍线性回归的矢量计算表达式之前，让我们先考虑对两个向量相加的两种方法。\n",
    "\n",
    "\n",
    "1. 向量相加的一种方法是，将这两个向量按元素逐一做标量加法。\n",
    "2. 向量相加的另一种方法是，将这两个向量直接做矢量加法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_bp6luds",
    "id": "631AD2C3EA1A431287E30A95D535D877",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "import time\n",
    "\n",
    "# init variable a, b as 1000 dimension vector\n",
    "n = 1000\n",
    "a = torch.ones(n)\n",
    "b = torch.ones(n)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_xxj5nbf",
    "id": "55B0FCA128314322808F46633FA9B944",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# define a timer class to record time\n",
    "class Timer(object):\n",
    "    \"\"\"Record multiple running times.\"\"\"\n",
    "    def __init__(self):\n",
    "        self.times = []\n",
    "        self.start()\n",
    "\n",
    "    def start(self):\n",
    "        # start the timer\n",
    "        self.start_time = time.time()\n",
    "\n",
    "    def stop(self):\n",
    "        # stop the timer and record time into a list\n",
    "        self.times.append(time.time() - self.start_time)\n",
    "        return self.times[-1]\n",
    "\n",
    "    def avg(self):\n",
    "        # calculate the average and return\n",
    "        return sum(self.times)/len(self.times)\n",
    "\n",
    "    def sum(self):\n",
    "        # return the sum of recorded time\n",
    "        return sum(self.times)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_g9h7dg8",
    "id": "2698821CF46844989522D09B8B1C76DB",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "现在我们可以来测试了。首先将两个向量使用for循环按元素逐一做标量加法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_eoz706b",
    "id": "DF2AACFBA2EA42698CC82C33AF79AEDB",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.02200 sec'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timer = Timer()\n",
    "c = torch.zeros(n)\n",
    "for i in range(n):\n",
    "    c[i] = a[i] + b[i]\n",
    "'%.5f sec' % timer.stop() # 参数 '%.5f sec'表示保留五位小数,如果配置特别好可以增加有效位数使得显示的时间不为0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_akkwkh8",
    "id": "B00F06B72BB5471DA82C945B04FED140",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "另外是使用torch来将两个向量直接做矢量加法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_a8sw68j",
    "id": "6D2503874A514A7590AF8F710B5F325C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.00100 sec'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "timer.start()\n",
    "d = a + b\n",
    "'%.5f sec' % timer.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_oonn3xx",
    "id": "B0CA3D998E0A4B5C848F9C1BAC37DB13",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "结果很明显,后者比前者运算速度更快。因此，我们应该尽可能采用矢量计算，以提升计算效率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_3y8h3t7",
    "id": "84D91561397548D7ACB5FAB71E66AB9B",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 线性回归模型从零开始的实现\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_3snj2zc",
    "id": "B3148881D9514B898929430997FD781C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.4.0\n"
     ]
    }
   ],
   "source": [
    "# import packages and modules\n",
    "%matplotlib inline\n",
    "import torch\n",
    "from IPython import display\n",
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "import random\n",
    "\n",
    "print(torch.__version__) #本机的版本是1.4.0,和鲸平台上的版本是1.3.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_ofruiuq",
    "id": "D7C96AC35B12411E8A1530B965CB34E0",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 生成数据集\n",
    "使用线性模型来生成数据集，生成一个1000个样本的数据集，下面是用来生成数据的线性关系：\n",
    "\n",
    "$$\n",
    "\\mathrm{price} = w_{\\mathrm{area}} \\cdot \\mathrm{area} + w_{\\mathrm{age}} \\cdot \\mathrm{age} + b\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_h3bosrm",
    "id": "1A5F9ED7F99643A3A440960077439F0F",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# set input feature number \n",
    "num_inputs = 2\n",
    "# set example number\n",
    "num_examples = 1000\n",
    "\n",
    "# set true weight and bias in order to generate corresponded label\n",
    "true_w = [2, -3.4]\n",
    "true_b = 4.2\n",
    "\n",
    "features = torch.randn(num_examples, num_inputs,\n",
    "                      dtype=torch.float32)\n",
    "labels = true_w[0] * features[:, 0] + true_w[1] * features[:, 1] + true_b\n",
    "labels += torch.tensor(np.random.normal(0, 0.01, size=labels.size()),  \n",
    "# 为了模拟真实数据,增加了符合正态分布的小的偏差\n",
    "                       dtype=torch.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_gr10soh",
    "id": "937B9B59AC2343B58488AAA9B7C11C2A",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 使用图像来展示生成的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_ov2af2a",
    "id": "8E2E1E16060241C6A33E4CF1EC65DF1D",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO2df3RU53nnvy8S4yJBsSTGKgaBQBIicopVW8YEQzC/smHL4rRbnDq7a477g/iPUOJme1pSTly3TtzdrusS754G2saVd5s0pk1rSuK2IBNAqICFS6iRAWmQQAgiRjOCIo3sQeLdP+68V3fu3HvnjmakmTvz/ZzjIxjN3PvOtf29z33e5/k+QkoJQggh3mVathdACCEkPSjkhBDicSjkhBDicSjkhBDicSjkhBDicYqzcdI5c+bI6urqbJyaEEI8y5kzZwaklH7z61kR8urqarS3t2fj1IQQ4lmEEFesXmdqhRBCPA6FnBBCPA6FnBBCPA6FnBBCPA6FnBBCPA6FnBBCPA6FnBBCPA6FPE3Cw1HsPRpAeDia7aUQQgoUCnma7G/vxSvvXMD+9t5sL4UQUqBkpbMzn9jaVBX3kxBCphoKeZqUl/rwxTU12V4GIaSAYWqFEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8DoWcEEI8jqeEnJaxhBCSiKeEnJaxhBCSiKfcD2kZSwghiXgqIleWseWlvmwvJQGmfQgh2cJTQp7LMO1DCMkWrlMrQohvA9gM4KaU8pOx134PwK8DCMbe9lUp5Q8zvUgvwLQPISRbpBKR/yWAz1q8/pqUsjH2T0GKOJDbaR9CSH7jWsillMcAhCdxLcQlzMcTQoxkIkf+JSHEOSHEt4UQZXZvEkJsF0K0CyHag8Gg3duIC5iPJ4QYSVfI/xRADYBGADcAvGr3RinlPillk5Syye/3p3nawmZrUxV2bVrKfDwhBECaQi6l7JdSjkkp7wH4MwDLM7Os3CRXUhrMxxNCjKQl5EKIuYa//gKAD9JbTm6TiZRGrtwMCCH5Qyrlh98F8CSAOUKIawBeBPCkEKIRgATQA+CLk7DGnCETJYbqZgAAX1xTk5F1WREejmJ/ey+2NlUxcickz3Et5FLKZyxe/osMriXnUSkNK9wK59amKkSiY4hERxEejk6ayE7VDYMQkn0KorNzKtIZbtMu5aU+lPiKsKela1KrTrghSkjh4CnTrIkyFdFpKmmXqegCdXp6IITkFwUh5LkmnNkQWebMCclfCiK1kq1yvVyqUGETESH5S0FE5NliMlM6qUbYNPUiJH+hkNuQiVTEZIpnqjcJ5swJyV8KIrUCpJ7mcJuKcDruZKZ0WJVCCFEUTESeagTrNprOVr02I2xCiKJghDzVNIcboQwPRxGJjmLn+jpGxoSQrFEwqZXJSHPsb+/FnpYulPiKWNJHCMkaBRORTwaFVgnCWnRCcpOCicgng0Kzk2UtOiG5CSNy4hr15LGhoRJ7jwYYmROSIzAiz2Gy3RlqPr96Ajnc0c/InJAcgkKew7hJZaQr9k6ftzs/a9gJyS2YWslhrDZTzRuOdnXs6n2PVZfj9Xc7sXtzA2r8MxPO4VQHb7eZyxp2QnILCnkWSVYFYhbM8HAUX3nrLI5cDALQhHdDQyVOXg5hQ0Nl3GeVQNf4SxEIDgPowBvPJY5Udaq8oWAT4g2YWskiVqmLZKmOIxeDWFvv14X3cEc/jlwM4nBHf9x7Vfrjj37pYayt92P35gbLNThV3mQ7R08IcQcj8ixiFQ07pTpU9L17c4MuvG7SH1aRuBsyZT+Q6fpz1rMTEg+FPItYpS6cUh0q+l6xuB81a2baHmOimAXSbi2pCqm6IUSioyjxFactwJxHSkg8FPIcw0mYJ9pJ6lZ4zQJpt5aJGpBFomNJP+dmrYXWUUtIMijkHkIJq8pdpxoRA87C61YgJ2pAFh6OosRXZPk5JeCR6Bj2tHQ6rpWbsITEw81OD5Jqq7zbum+3lgNGIU1lM1Sla/a39yZ8ZvxmI1mjTkiKMCL3IJmy5E1309Ac6bs5nt3TgfE7cQOTkNSgkHuQiaRYzCJrVZOeKuYbipsUTipNRqxOIcQdTK14mFRSLOb3WtWkW5HKKDurFI6dX4sbYW5u68Yr71xAc1t30vcSUsgwIvcwqaRYzO91m8pIpULFKqpOr1RQmH4SQqygkHuYVHLf5ve6rfxIt9TPzkLADdtWVttWuWQapnGIl2FqJQ9JtfXfCatUSCrHsrMQMGJ3vKkc3MGhGcTLUMizTKb9TOwGQicT91TWkYroqby5Gkahjm88nzreV946i/evDOK5N04jEBxKuiY3a3b7vWjNS7yM69SKEOLbADYDuCml/GTstXIA3wNQDaAHwNNSysHMLzN/yXS7uRoIvWvT0rhINpmvCwDX60gl3aKi6r1HA3HHN557a1MVTl4O4cjFIK6GI7pb44rFFY5rcnPt3F5fNhkRL5NKjvwvAfxvAG8aXvsdAC1Syj8UQvxO7O+/nbnl5T8TzUEHgkN4+WBHgs94KuV9Vu9NRZxTYWtTFSLRMUSiowgPRxM2W199ujHOP33HujocvXQz4cnCmMs2H9MqBcN2flIIuE6tSCmPAQibXn4KQHPsz80APpehdRUME80Dv3ywA0cuBvHywY4JH88oyJO90acdV2JPSxea27oT1qn+/sjCMrzx3HK81xPGnpYulPiK4tZkTOuUl/pQ4ivCnpYu2zRPoQ3IJoVJujnySinlDQCI/XzA7o1CiO1CiHYhRHswGEzztGT35gZHn3GFXR7c+Ofmth688s4F/MZ33085V59ant19OaFdbn1DQyXW1vv1KhjmtgmZwvJDKeU+APsAoKmpSU7VefOVGv9MVz7jVnnwk5dDWDZ/Nva0dMVe1f51tHaF0NzWE/u7wLaV1Y6RrLk7VB0fsM5HW5UT2pX9qUj6tUMXsaelC5HoKF7YWJ9g5cvcNiHpC3m/EGKulPKGEGIugJuZWBQZJ936ZnOOWG0qLpt/vymSFdAEXOoCX+IrchRJu+5Qu+h4Yg1D41G8XUUOIYVOukJ+AMA2AH8Y+/l22isicaRb1WIWT7WpaL4xvLBxiZ5m2b56EWbEBkA4YTWxaCKboMafZoxRvLkix3iTAyY/z09IrpJK+eF3ATwJYI4Q4hqAF6EJ+FtCiF8FcBXA1slYZCGT6aoLp1SEJpSd2LVpqSsDK6uJRalgFmJlAKbWos6l1uJk0hWJjsalYAgpJFwLuZTyGZtfrc/QWogBJXITaW13c1yjIJvPtbWpKuF94+PaxlDiK8KGhkpEomPYub5W35C0i4bt0kPxQqwNlFDHNz+FWB3DKOxabh+gLwspROi1MkVMdM6lymkDmWsYMnuIW9nZmht4xse1jcata9empTjc0a+/9urTjQnfz40H+bjDobSsD29u606IuI3R+lT6shCSa1DIp4iJzrnc0FCJFYv7MyZQVukJpw1Lc3Q+GIni3LXb2LGuDisWVyRsou5v702IolWkb47c44V4UezM4xG1tukq8MLGJaATIiH2UMiniHSm+qSSf04W+Ztz5FYblub3GaNzALG8eIXtJqqiua1HT5e8sHFJQpRvXleJrxivvHMBJb4iqJJI9TNZ6WKmrQ4I8RIU8iliquqdUxU0NxuWxqeDA2evY+f6WkcLACWwI9HR2G9l3HHsbmbm35f4irGhoRKvHboEQGLbykW2OXa24pNChkKeZ1gJmlOU7kYAjcZXqqrFzTCKnevrsHN9LVQNeDLMNwMAOHC2z1DXXuw451NF5ixBJIUGhTzPSLXpxs2TglVVi917lJmVep9KrwCwrEaxQ615VW0Ftn96MWZMn5bUCCwT6RUOmCBehEJeAKSbdkgmkFaVL+PvG891200LMt4oDnf06zeD451BtHaF0DD3p1Hic/5PVev6HLNM+zh9prmtByPRUczwFWPbymrm2oknoZDnKeZmGwAYjLiLNs1RabIbgbnyxfj5bSsXoSTWJareZ8zHG28CxlJL7VxahUrHjTto7dLKE1XJpPl7GJuZADjWtSuMTwsAcO7aLd2EjLl24iUo5HmKnVmWm5p08yZiMvE356pfO3QJe1o6ERqKYoZvGpQg2w23UDeB3Zsb9FLL/e29aO0a0F9XkTqAhJpyowfLhoZKy7p4a7SnheXVZfAVF+k3GUbixGtQyD2OXU7XSjRVTXqyTkzjZ92kVRLPrwlkx43baO0KARg34HIablFe6kPNmplxaRJVqaJef+3QRZy8rGzxtRuE0YNFVeGY6+KtMD4tqOMwEidehELuceyE1rwRaKxJd6rnNn82WV7b3FpvTKdo5Yp9GLl7T+/SVGu2agoCtMlH299sRyA4nFAdowQbAGr8pdjS+CAA50lHTjcsu2tk9T25+UlyGQq5x5nIRmYqn7GrMx8vMazFrk1L9dZ9IH6z84WN9fjGDzqwp6ULI9ExVMy8L8EiwCiUX/v7DxAIDqO6oiRhfVrr/ijOXBlEa1cIB8726RG1lSAnu2G5wcrSgMJOcg0KuceZSKOR02fMFSR2JYcqUt/SOA81fi3tUWJjfdtx447+86WnFsRF+GahfGjebJwIhPCZh35GF8pAcAgvvv0BGubOxvNP1mBL4zy8fLADI3fvYU/LeC6/ua0bI9F7mOErwraV1RO6ySXb6GVVC8lFKOR5QCajRCVUe48FEB6+q6dMzCSb1GNc00tPPaQPijZ/ziyUz6+pwYzpWou+SsWoVEtrVwgVM32IREdx5GIQwx+PD5kwpl2A8Zy8m81aq1b/k5dD2LGuDq+/2xlnX5CJDlKt7LEbbqYwEeIGCnkekG6UaG7meau9F4HgMGr8pQCk5bGdcufNbT04eXkAp7oH9coS5cWi7G8j0VEEgkN6NYpxCDMwbpgFSASCw5g9oxhPNc7DhoZKvPj2eQDA6Z5BrP+Edv5IdAzbVy8CIDDD4MlidW3sLHoBTaBVdc/VcASB4DCADn2sXiasFow3nWRTmAhxA4XcYyTz5Z4IZrHb92yTHkGXlfgsUyZOuXNjbbaxskTzMx/FuWu3ceRiUP+pzqtQHi2qUQcAbo+MYt79M3C4ox+tXQN4fFE5iqcJbGio1M+5c32dvuFqd22smpeMtrnAuAHYY9XlekSeSVSuHxCskiEZgULuMawizHSjRLPYmQc7Wx3b7uahRHEkOoYZvmnYtrI67n2R6Jhl3bjxBqXEW3VbAsBITGi3NM5DJDqGM1fCsQ3P6wAkdq6vg3p6CA19jI4b/67n1M1t/Or8xjJMZR9w7tptvPp0o/4ZNwOuU6W81McpRiSjUMhzgFRy3OlG31bnSmfDNDwcTfAY1/zD7d9vjJpVNG+sMDFa1qrjqd+fuTKIu2MSp7rDWFU7B2pYtBokXeIrxvHOAbR2hdDaFcIMX1HcscZvNKN48e3zaO0a0K+nlad6Jq4vIZMNhTwHSCXHnW70nemqC6fjpXLTMObcrSxxH6suR42/VG8wAoCGubMACGxfvQiR6Ji+Bi2PrlW5jMTq3EPDUVTEhLzEV6Snf2r8pfo5zZ7q5vW7EWlWtZBsQCHPATJRCZGNcwWCQzjeGcT21Ystj6da6UNDUVTM9DkKYLJ69bX1fgSCw3h8UTkA4OH5szHDV4w9LZ2o8ZciEBzGuWu38OrTjajxz8Q3n3kE+9t7ERr6GABwvu82TgRC+nePREdx8nIYp7rDOHC2D9tWLrL0bzGKshuRnsp/l4QoKOQ5wFQNncjkucLDUfzqX76HnlAEgLARaGV6pbXqv9Xei33PNqHGnzjAwrjhqOZ0qtcBLWJfNr8PZ67cQmvXANYtfQBbm6pw7totHLkYRI2/FEcuBtHc1oMSX5HecbpzfR12bVoa56yoctSvHbqEU91hjETvWfqzKMGPRMcQHo46irQxWmckTqaaadleAHFG5aDdDGaYSva398ZEXKU4Etm2shq7Ni3FS099Uo+aXz7YEfce9f00tHz3+CDm8RtPjX8mSnzFupGWEuRXn27EzvW1WP+JytgQC1UuKbFr01K9jd98ztcOXcTIXc3PZYavKMGfRQkzILCnpRP723v1tVjdtFS0rn1mYuTqv2uS+zAiz3FyNeeqImg1gs0KY/RvLGk0or5faCiKfzr/k9ir1gOWjec0osoY1YanOobq9tzT0oVjl4I4EQjh5OUQls2frddxa8Zc1Qlli2YLgmSpEpXjf6y63NbfJVmOPdX9BkIUFPIcJ1dzrnbVKUYCwSFdvI0ljVbThI53BnElHEGNv1QvObQ6pyoTVGPfzF7o5aU+jETvYd/xy7ESSK0r9aF5s+ErnoYjF4Ooq5yFheUluBLW0kKAVhbZ3Natuy2aXRmToXL8AGwtdJPdlJ3+XefqDZ3kBhTyHGcq8+eZ5uWDHTFR64irxzY2B6lmow0NlXj5YAd2rKuLqxxxan4y+pBvaXxQr255K5be6LhxG9985hE92lbHi0RHcSUcwdp6vz4VaHwcXbGePnE7As94Q1JWwebaeDcDOpzOmas3dJIbUMjJpKGlURLTKcbmIGOU+cZzy+PqyQFYNj8pT5RIdBR7WrrwRE1FzBFxADX+UtwauYsZ06fh11Yt1o20mtt6dCOt5rYe7Fxfiy2N8+JsAwCZklCao2RVJx+JjqK5rQcA9BuE25uDHV6+oZPJh0JOJg1zhygQ76544GyfbnqlMFaKGP3GjWmawx39sfx1HZ6oqdDLCtfW+7FjXR1+629+jEBwGL/3D+f1DVkAuvmXGgl34Gyfnj9//QuPpJx7tqq0MfqouM2vE5IuFHIypagoVhlzmYdHaHnwYj3SVeJrTNO8+nSj7otS88BMnAiE8Piicrz6dKMmps+vjCsnfHxROVYsrojdPK5j++rFiERHMXL3HgDgRCCE//T6ccy7fwZe+c/L4sojnTYZrXL2Rh+VTDobJtvs5GZoYUMhJ5OKVZ5YtcJblfupfPnJyyG9KzMSHUNd5SxER+/plrJK7FfVVgAAHp5/f9x5dm9uwN2xcQ/z8lIf9h4NYE9LJ1bVVqC1K4Ttqxfpf+679RH6bn2Elw9a5/MB+w1KY635ZPmoJJujys3QwoZCXsBYRXGZjuzMAmNuhbcSIkCr/Fg2f7Y+fWjfscvYtWmpHi0rsd+xrg6r68IJE4oOnO1Da1cIjy4sS9goDQSHAAAfjd7DN595BN/6URfarwyieJqIy+drnasD2L56keMGpbqpJLOkTefaGjc7rUSbm6GFTUaEXAjRA+AOgDEAo1LKpkwcl0wuVoLgxr87FcxVJsYqj2QVHerPanPS+Lvxlv4KbG2qwp8cvoSF5SV4rLo89g6h/zR3XX5+bxsA4OJP/h3lpT58fvkCdN4cwo51dTjc0Q80AAfOXsfbZ/vQE4pgepFd56p2bUJDH+PxReUIDX2sR+XmqUZm3/NUx8YZNzutrhU3QwubTEbka6WUAxk8HplknMTT+Fo6jSpGgbGqSDl5OYRXn27UXxuMRHHgbB8AgcFIVM+N79q0VD+GMf2i/Mjf/JcrAIDX3+2MHU+ztlXlhcb1P1xVhlPdg6iv/GnsPRpAy4f9ON0ziK6bQ+gdHMG7F27iVHcYAFBdUYK6yllxtgHG727MxZ/qDqNi5n3Y2lQVN9Wo8+YdvPp0Y8K1NU4jUvl9ddzmtm6M3L2HGdOn6bXtVteUEICplYLGShCsXstEo4pWlqe1xKtNx1W1FbptLDAu7EoYlY+KyqU3t/XoOfOSWFu9qtnuGxzBsc4gdqyr07s5d66vtYz21Ti59p4w3jx5RTfiGrundYz2DY4AAJ6oqUBTdTn2tHSiwuK6qGakVbUVaHhwNmZMn6Y/aagB0uWlvpgHTDde2Fgfdww769zEsXXFFG7iSKaEXAL4ZyGEBLBXSrnP/AYhxHYA2wFgwYIFGTotmQpSaVSxi9BV082uTUtxuKM/ZmhVi0cXlusDI4BxcyxAYEvjg1g2XxscoTH+03ze3sEIekIRvNcThkqrjETv6RG8OZVT4ivCiUBIL1n8zbfOoicUQXVFCXpCEV3EtzQ+iBJfUdwQCqvmJON3Nb7e3NaD96/egjnFo7zbVa29cWSe2kAduXsPkNDLG9V1VJ9npQpRZErIn5BSXhdCPADgkBDigpTymPENMXHfBwBNTU3S6iDEe5gHTKgmHQAJ0afxp/qziuiNUaeylNXGzI2X921buQglvmI9naIEbO/RQEIVjHJANKdyjl0K6gJtXIOqN//MQz+DilKf7p547tot7N7cgK/9/Qc4EQjpM0iN393ummjfZdzHxerpxcq+11j5otJRJbGpSXa2ulaVLKRwyIiQSymvx37eFEL8HYDlAI45f4rkE+MmU3WWTTBm0TNv3BkjXrNAGX8C2kbknpZOPbes8uWq4sTYcLT904v1iF+lMU4EQnqFSXg4itBwFMury9C4oAzPxyprwsNRnLkSxpGLQXQPvGdoLLI29DJijpSdNimTVZu4+YzdTYIUDmkLuRCiFMA0KeWd2J8/A+D3014Z8RROJlPJmmq+uKYmbiPU6CSoRNnYkr9zfa2eX1f+4yqqBeJz7Wvr/fqg592bG/S0jaqiMW5WfqqmIm6S0aMLy9HaFUJPKILHF5VhxeI5cYZexu8FIGa/K6DseNXNxeiD7mZPwun3xj8nu0mQwiETEXklgL8TQqjjfUdK+Y8ZOC7xEE6ClOpkHbWJCGj15KoL1Bzta2PfEvPlwLh51YaGStwd+yBWl96HFzbW6wIciY7iyMWgnhc357G3raxGe08YJwIhFE+bpndqqveEhqLYd/yyPmZuvDW/Tr+BdN08jd7BEfQNjmBe2Yy4ARfq2piHXtjh5oZICpO0hVxKeRnAwxlYC8lT3ESLVtGltgHYoUfWxpZ3lS+3inTVn2vWzDQNadDSIuY0UH3lLPzBDzqwZok/4abz+hce0aN2JaLq748vKgMADA5H9SajspLpWLNEW6sx2j/WGURPKBJXlQMkVuo4iTH9yokdLD8kk06q0aLx/VZdoHbHtBKz/e29aO0K6TcCIDEN9NwbpxEIDuOPD13CowvvjzPyMleWqKeFJ2oqMBorVzx6SfNSLy+djvDwXbz+bifeeG45Xn26Ed/6UQAdN27jNzfW472ecJzNrcL8mjllY7bKTacMlOQnFHKSE9hFlKncBNy0rqsNVbXR+dqhS6gqm4FVtXPQMHcW9rR0YW29P25NkeiYbtilNlRVVUt56XTd23zHujq8/m6n/h6ta7QKLx+8g9kl0+OeFBTqtbImn/79zXYFZqtc8/XS1jOa4CSZDEbx+QOFnEwKqYqEXUSZynGSta4bN1RV+Z7yC19VW4GP7t5DdUUJjlwMYsd33ocQ0M211HDnFYv78cU1NQgEh3Dw3HUEgsNYW+/Xu1OXzb8fB8726ePv1IDqu2Pn8f9+7fG49SprXnUDUOkV4/cYjET1DlbjNdnxnff1csgSXzH2tHTpTpJurxmj+PyBQk4mhVRFwi5tkMpxjEMnrDYQjW39agDE9k8vxvm+22jtCsU2T4GF5SW6x/mq2grM8BXrgr21SfNGVwK9qnYOdm9uiKuqAaDXfTsNqFb2A1fDkbjjG28+KpVjrDPf396rr09V4BivnVu/HFa65A8UcjIhkkV9qYqEUbysRqgl6xxVGP1LjlwMxo2TO3C2T69eUVHs2no/fv9zn8SBs304eTmMU91hrKn349glbXPy0YVlCcOZvxLrAgWAxf7SuEEX6sawoaESZSU+ffLQlsZ5CZ2hKve+Y10d/vH8T3C+7zYGI/GeLnZ15GbPc3PzldXAC6tpSxN9+iG5BYWcTIhkkXI65XBWI9TcntdY8bJicb+pu3PcEdHoc6KGVaiO0kh0FD0hNdNzUUKkX/fALLx/dRC3R0YRuDmEL29Yop9bRcsvH9SOqQZUm9M6QPwEpdff7dQ/Z/RDt6s9d/I8txt4Ybw+bq478Q4UcjIhMvlYnsqQYrfnLSsZtw4AtM3ANUsewLlrt7Cl8UG9GuVquD3OtEp9xljaCEA34nr3Qj9OdQ/i2RUL0TsYifMvH4xoxmCPLLgfRy4G8a0fBfDVn/+Eq3Xv3tyASPTfcHskim/84EN8fnmVZW2526jZqmM2mTgz1eJdKORkQmSyAcVq+ITdsZOd1+pYKjo9d+12XL75wNk+BILDeHxRWdyEH+M51IZkVXlJ7AxaVF9WOh1f3tiI5rYenLkSRmtXSG9cqiqbAQBovxKOS6c4+aHU+GfGOlTDeP/qbXTevBO3+ak+p24oRs8XM0axT2UTmU1F3oVCTrKOVfToNkebLJo32uduaZxnquHWRHl6UZFevTISHUPHjdt46alPosY/Ey++/QFau0J4fFEZdm1aqlvwAlK31QWA8tLp+oZlVVkJ3jx5BdOLptmaXJlFtbmtB/5Z96GqbAY+tbgC95f4sGz+/XHifbwziIa5s+PWbnUtVGmk8Tqks4lMcp9p2V4AISoSVJuGyp/ciHJXjO/UHBck9Rl1LGND0J6WTpT4ilHjn6n/LhAcwsnLmkD/5sYl2Lm+Du09Yew7flkrOXyzHeHhqC6cD88vwxfX1MRGzWleKiOx2u3HF5UjPHwXD87+KUSiY/jcz83Drk1L8Y1f/Fld/PceDeCx6nKsrffHlRIC0G8Ib7VfQ+/gCIJDH2Pf8cso8RXFvocm2qqqZm29X3dvtLoWgNStDMzXQ7G1qSpubebrSrwFI3KSM0ykc9Gte6A52n/5YIc+BegfP7iBlgs3Y2mWcvzk9kcIBIexv70Xzz9Zg4qZvrgI/8yVQQDADF8xXti4BK8duohT3WFcv/0Rrt/+SO/sBLTmH7XRqfxXVizu1xuANjRUor1HW8fPVc1G6X3Tse1T1fqaAcQ6UiWUoJvLEY3fVVXJuH2qUU6S5utKvAWFnOQMqQywcMLKRtZcNbJ7cwPujp1Hw9xZ6LhxB4HgMGr8pfjT//ooACR8XtHc1oPWrhBW1c7BmiV+PPfGaexYVwcAGBy+i8sDw3EboMY111fOwtVwBI9Vl+tR+PHOAZwIaMcDJFq7BjC9SMTKJMYImT8AABUzSURBVK/rZY8qHx4IDuHctVt4rLoce48GEurlzdUqdoz7zdRa2g7bXUuWKOYmFHLiCexEXuWPQ0Mf6/MyraJ3YzMQoG0uqk5LtaG5e3ODLk52jTQj0VEAwOI5JXjhe2dxJRzB3bF7+OYzj2B/ey+2PVGNwx1axG0Wuj9v7UYgOIzX3+3EsvlayqZh7iysrpujNxOtrffH7Hav686LarTd1qYqfRCFaiIyG25taKjE8c4BhIY+RiA4ZOuqaPabsUNdS1WPbzc4xA28CUweFHLicbR0Q8eNO2jt6gZgPX5OdVFapSSM9dxmzDeFGbGOzcsDEVwJq67N2foN5bunr+rNQkro1O+eXbEA04s0odamHxXHRbrKy1xNRlKj6AAZN2jD6LWungbUDepwRz9auwbQ2jWAzptDtq6KbitUxr1cxhwHh7iBG6yTB4WceBrVdWlOMZibiJTveGgoqpcZusF8UzCe78BZbUjFtpXVaG7rAQC9kShe6LSbzeWBYXzzmUcSon4gvolHDcEwntco+soRckNDJV58+zxauwawbP51vLBxSVyefEvjPCyb3xdXWpkqxlF+xu7WicA69clDSDn14zObmppke3v7lJ+XFAZWed3f+O6/orVrAACwa9NS1xFhIDiEr37/HACBb/ziz8aqVsZ5/8ogfutvfowvr6/D99qvoWHuLHx++QJ87/RVdNy4g5eeeghlJT7dm9x8blV6qIRXPTlYrVF9L3XTMpYZLq8uw6dq5sR5tgPjHaWpfGeSuwghzkgpm8yvMyIneYV5fJsqvXt04f1o7RrAEzUVjhGhWSyPdwZxqlurUlHt88YbxW/9zY8RCA7jxX84j/DwXayum4PDHf3Yd7w77jOvPt2I5rbuuOjYvFZAxPmvm9cUGvoY+45349ilYGyDtALbVy/G2d5BnO7R/lGzSBVuomDmrr0PhZzkFSqNYk5vmCcKOX3eaLr17KcW4kooAv+s+7BjXR32Hg3Ebfj90S89rEfk3//XPt0sKzT0MX587TbqHpipC3eJrxivvHMBZ66E8ehCbbrQkYtBLCwvwZVwBO09YWxbWa2nMszDqLXKFuChebPhK56GIxeDWF3nx6dq5uB0zyBW1c5JEGw3uXDmrr0PhZzkFca6cWOUmerm3rjp1ih6B0dQ+8BMHL0UxJ6WzrgNv/JSH1q+8iT2Hg3E+ZV/9ecb9LRGxcz7dOOq450Dsc3IkH6cDQ2V+Nrff4ATgRCa27rxwsb6uFZ85W1uNe8zXrglegaG8ZW3zmL35oaENFCy78zctXehkJO8wq5uPNXPA1ozjzZA4kasrnt2nIAD8akYINFu1vjTnOIx5rObqstjHuPjDo3qp3lNCuP3UhulB8/dQCA4DCDeRZGDm/MbCjnJC1JxUHR7DEAr51MeKlsa5+FwR3/cZ1JNS6jo2uyVMt69qa3D6IGeLIethmTsXF+HWn8pXvyH83p3qNt1ujkH8+i5C4Wc5AWpOCi6PQYQH1Ub8+evPt1oa9Jldh40Drcw5soBxJX0qdfVhqXdiDqrdatRb9//1z6Eh++i+V968OTSB/T3JLuxJRN69fu32nux79kmx7QNRX/qoZCTvCATeV6rYxhvCMZmHOVfbsYoiOo4oeEo9rRcwLsXbmLF4grd8CoSHcUr71zAsUtBNFWXY0vjg4hExxAa+hivHbqkp17c+slsbarCY9XluBqO6I1CVt/DzXe3esJRNr0vvv0BVtf5bYU61acUCn/6UMhJXuAkVE5CYeXLYvcZczOOuYJFtcgrKwB1vNcOXQQAnOrWRsmpiDs8HNU90k8EQijxFaHEN26pq6L1ZCJnnvEZCA7jvZ4wHolVxjhdD/UZ8+AJqyecfc824eWDHah7YJarKU1ub6qsmkkfCjnJe5yEwu5342mRsbj0h3kz1dyyrrxQjFYA21YuQnvPoF77bdz8VPXlI3fvIRIdxZbGefo8TqfBEHa4EVHjMQFYDmpWHu7G49T4Z+rr3bm+zvYcqaa1WDWTPhRykjfYRZpOaQM7ERn3GBm1FHRjmaJxA1S1yBsHH5eX+vD6Fx6xjKzV/M1v/KAj5nE+huefrNX91ZOlPKwIDUfx/P9tR+OCMjxv40Wu1n7gbF+CKCsP912bliZ81piPz1QahFUz6UMhJ3mDU6TplDZwGh6t5ncqQTdudFqVOdpZySYTq44bd/SfTutLNuptf3sv9h27DAA43TOIzv47+nqBxM1YK1E2O0UaSyytInWSfSjkJG+wiq6tBCeVR3mjoKt8dnNbjx6dO50zFbF76amHdCtd1Rl6vHMAGxoqTRUi5jrzxO8WGo7i7NVBCCESNmatNmPN6zSnh8zdrpmMxklmoGkWyXmmuqrB7nzjMzFH9Ug2UykB8zmfe+O0bjWQrLHHbviDuRP0W0cD+HHvLTw8fzaef7I26eedjpeJfw+sVkkdmmYRzzLVVQ3mqNVc1aLSLammF5wE0/wdNRvbjrhpQ3bCZ/6s1fV67dAlPeUyvWia4+eN31P5vVh1lk70e6vXzOZmZOJkRMiFEJ8FsAdAEYA/l1L+YSaOSwgw9VUNVk1AABzz3ZodbTeUP7mb+mqnNIdx2IX5ScC4FqvPWl+v8Sfvhrmz4tatukLN1zdTN1Cr49iZm5GJkbaQCyGKAPwfABsBXAPwnhDigJSyI91jEwJMfVWDuQnI+BOwjjDVxiGAOCtZ43vNm4jGYzt9RyWE21cvxhM1FQgNxw/HUFU0dnNG1fShZ1csxOWBIXx++QL9d2rzdOf62oSbT6ZuoMn2EZhWSZ9MROTLAXRJKS8DgBDirwE8BYBCTlyTq/lSK4FVg5Mj0TG8sHEJgNgm41AUHTdu60INJFbSGDcR3d6g1A0AAE4EQjgRCGHG9CJsW1mtXzO7ahZjCmNtvR+tXSEc7tDOHx6O4syVW7F3Jm6eZuoGanUclhxmlmkZOMY8AL2Gv1+LvRaHEGK7EKJdCNEeDAbNvyYFjhI8VT+d20jTT02YKmb6dKFUbG2qws71dYhER7GhoTJh3mUgOITn3jiNQHBIf03lprVIeryKZIavCE/UVOjnbm7rxivvXNBTOhrxgqxSGDX+UuxYF9+8tL+9F61dA1hb74+ZdhGvkomI3KoOKqEURkq5D8A+QKtaycB5SR7hpe4+45AKI3ZeLaquHNBmbxpRo92uhtux//mVKC/1JeSUjcdVUfiGhsrYCDrg5OUQvvGLy2B0T1RPNVubxj3Qj14K6k8Q5vWm8xSUq09ThUQmIvJrAIz/Rc8HcD0DxyUFhHrU9oIQ2K3V7vWtTVXYtWkpAJHw1KENgChFIDgc181pjJyNx1V/PtzRr4+gO9U9iMMd/SjxFWNPS2fc8ZUHukZ8/GQ8rvkpIBVSfZpK51zEmkxE5O8BqBNCLALQB+CXAXwhA8clJC+wmkSvKCvxYfOyBwHIBOE2EggO6Q1DNf6Z2NpUpc/urK4o0UfMAYlPNXZPEEaSdYw64fQ0ZbcxTJOszJJ2RC6lHAXwJQD/BOBDAG9JKc+ne1xC8g2rCFgT0E6U+IoTmo+MUatKwbx8sEM/1utfeARr6/3oCUVwuKM/5ScFYDxHPzh8N/aKdceo3bqSHd8qWjc/caRyLmJNJlIrkFL+UEq5REpZI6X8eiaOSYiXMYqQlSCNR6XCUtTMArh7cwPW1vvjGoSUe6KdKCZbAzB+gwgEh7Br01JsW1nt+LlU0yhWou02jeatDfDsws5OQiaBZFax5o1GYxel2V1RvW5s1QeSbzIaJxQpnxjjGgDEdZAqTxdlBHbycgjL5s+Oa0JKdVM6nTJDL22AZxsKOSFpYCemViJkFGWzwFm1yW9tqnJsY0+Wax634h2z7aI0dpAaP6cMspbNv99VGsTpWpDJh0JOyARx8gsxC7XZ8tbciWkl/Mna2M2dompN1t4wRY4Ca/6cmoSUzNfFyS/G7TW0E39uirqHQk7IBLESWidhcvJwsUpBGNMrVse0mkZkNSDaqoVfMe7jMqaPmFPvNZ83PBxFaCiKVbUV+s3DjS1usmtoJ9ZMrbiHQk7IBLFqqHESpmQeLmashlcYBdnOw8Q4IFq932y4ZRbwnetr9RSK2X9cfWZ/ey/2HdccFFWbv5NfjJtUi9N1YBu/e+hHTkgaJPPyzvQ5VL339tWLUDHzPsvzWKU7dq6vi0uvjM8crdVrzM3+5Gb/cSeHR6vvbZxrapXaYU49dehHTsgk4CZF4hY7YYs/plbn3XHjDlq7ugEk5tvtIn+7zVi7unMg3n9czRd1cx2M51Bj8oy/s/sMmRgUckLSIJN5XDfCtm1lNUp8RbHBydcRiY7qboxWn7O7sTjdcCYSKdv5zDgN4tjaVIVIdBSR6FicPwxJHQo5IWmQyTxuKnlzALoZlzG/nQkmEik7XQenm0mJrzg2qLqIUXkaUMgJyRFSjZInazjDVFaLsDIlM3CzkxAPoDYOUx347GYEXSpox+sBILFt5aKMp0O4AeoMNzsJ8TATjVztRtBNFO14nbHjFadcbujm+BPdAC3kmwCFnBAP4DYXb2V3G4mOAhAZSV9oxxuD0XZX4TRuzq3AppNqKeQqGAo5IXmEcjMEOvDGc8sdSwaB1KNY7XhLEl53mv9p1W3qdPxMm2wVQqSeERtbQkhqTJbXtpXdrRNOVrGprNFp/ufWpiqsrffjyMUgmtu6J81j3M4etxDscBmRE5IhUon8JisNoNwMzba4dljZ5bqd5GP8vskajJQJVyQ6NqHvnU5UXQiVMRRyQjJEKuKcCXGZiHOg+TNWfi7Gzkynhp1xv/OxpO6KTuPu3JDOja8QPFso5IRkiFTEORPiMhHnQLvP2HVmmht2rKJwuxZ8Kyb6vSf7xud1KOSEZIipjvwm4hxo5WFu9/7wcBSR6Ch2rq/Tz2GcOmRsu0823DldjBG9m5SRFflc1UIhJ8SjTOTGYeVhboeqQd+1aWlCN6k5152JQRJuSEeM8zlXTiEnJM+x25RMRnIjrKnNddutyS2ZNgrLJSjkhOQ5ZvF0ElA7L3Iz2cp1T1b6yutpFwo5IXlOKuJpNx0oUxFrrlaQeD3tQiEnJM9JRTyNdeUrFvcnbHICU++BMhVpj1y9wbiFQk5IjpALeVq76UCZ8kAxTzNK9fOZNunKFyjkhOQIuZynzZQHitN3tBPmVGviCxEKOSE5ghfytKlEwcb3Ws0QNWMnzHY3ES9cr6mCQk5IjuCFPG0qUbDVe52+Y6rCPJHrla/pGAo5IcQ1E6lDtzLksmIqhDlf0zG0sSWEAHBnW2tnFev03sMd/ZNmI5uqRe3WpqqMDqrOFdKKyIUQvwfg1wEEYy99VUr5w3QXRQiZetxEqxNJTUxmLnsq0jFeIBOpldeklP8rA8chhGQRN6I4kdTEZIpnOsfOp3w5c+SEEADuRDGfKkXyKV+eCSH/khDiWQDtAL4ipRy0epMQYjuA7QCwYMGCDJyWEDLV5FNqIp9uSkJK6fwGIQ4D+BmLX/0ugJMABgBIAH8AYK6U8leSnbSpqUm2t7envlpCCClghBBnpJRN5teTRuRSyg0uT/BnAA5OYG2EEELSIK3yQyHEXMNffwHAB+kthxBCSKqkW0f+P4UQ/yaEOAdgLYAXMrAmQkge4aY+naRHWpudUsr/lqmFEELyk3yqDslVWH5ICJlU8qk6JFdhiz4hZFIpL/XpFrZMr0wOFHJCyKSTqicKSQ2mVgghkw7TK5MLhZwQMunkU0doLsLUCiFkUvBy2aHX1k4hJ4RMCl7Oi3tt7UytEEImBS/nxb229qSmWZMBTbMIISR17EyzmFohhBCPQyEnhBCPQyEnhBCPQyEnhBCPQyEnhJApYDJr0ynkhBAyBUxmbTrryAkhZAqYzNp0CjkhhEwBk+k3w9QKIYR4HAo5IYR4HAo5IYR4HAo5IYR4HAo5IYR4HAo5IYR4HAo5IYR4nKz4kQshggCuTPmJE5kDYCDbi5gAXly3F9cMeHPdXlwzwHW7YaGU0m9+MStCnisIIdqtTNpzHS+u24trBry5bi+uGeC604GpFUII8TgUckII8TiFLuT7sr2ACeLFdXtxzYA31+3FNQNc94Qp6Bw5IYTkA4UekRNCiOehkBNCiMcpeCEXQvyBEOKcEOKsEOKfhRAPZntNyRBC/JEQ4kJs3X8nhLg/22tygxBiqxDivBDinhAip8vMhBCfFUJcFEJ0CSF+J9vrcYMQ4ttCiJtCiA+yvZZUEEJUCSGOCCE+jP33sTPba0qGEOKnhBCnhRA/jq35payup9Bz5EKIn5ZS/nvsz78BoEFK+XyWl+WIEOIzAN6VUo4KIf4HAEgpfzvLy0qKEOITAO4B2Avgv0sp27O8JEuEEEUALgHYCOAagPcAPCOl7MjqwpIghPg0gCEAb0opP5nt9bhFCDEXwFwp5ftCiFkAzgD4XC5fbyGEAFAqpRwSQkwH0Apgp5TyZDbWU/ARuRLxGKUAcv7OJqX8ZynlaOyvJwHMz+Z63CKl/FBKeTHb63DBcgBdUsrLUsoogL8G8FSW15QUKeUxAOFsryNVpJQ3pJTvx/58B8CHAOZld1XOSI2h2F+nx/7JmnYUvJADgBDi60KIXgD/BcDXsr2eFPkVAO9kexF5xjwAxgm515DjwpIvCCGqAfwcgFPZXUlyhBBFQoizAG4COCSlzNqaC0LIhRCHhRAfWPzzFABIKX9XSlkF4K8AfCm7q9VItubYe34XwCi0decEbtbtAYTFazn/pOZ1hBAzAfwtgC+bnpRzEinlmJSyEdoT8XIhRNbSWQUxfFlKucHlW78D4AcAXpzE5bgi2ZqFENsAbAawXubQRkcK1zqXuQbAOOp8PoDrWVpLQRDLM/8tgL+SUn4/2+tJBSnlLSHEjwB8FkBWNpoLIiJ3QghRZ/jrFgAXsrUWtwghPgvgtwFskVJGsr2ePOQ9AHVCiEVCCB+AXwZwIMtryltiG4d/AeBDKeUfZ3s9bhBC+FW1mBBiBoANyKJ2sGpFiL8FUA+tmuIKgOellH3ZXZUzQoguAPcBCMVeOpnrlTYAIIT4BQCvA/ADuAXgrJTyP2R3VdYIIf4jgD8BUATg21LKr2d5SUkRQnwXwJPQbFX7AbwopfyLrC7KBUKIVQCOA/g3aP8fAsBXpZQ/zN6qnBFCLAPQDO2/j2kA3pJS/n7W1lPoQk4IIV6n4FMrhBDidSjkhBDicSjkhBDicSjkhBDicSjkhBDicSjkhBDicSjkhBDicf4/pzb/XVawJGYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(features[:, 1].numpy(), labels.numpy(), 1);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_iivzo2j",
    "id": "27981A0FD4054AC39194415A90F313EC",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 读取数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_0tj7eus",
    "id": "A6E1419DA00C4ABF8CBF0E0F0B2B9E35",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def data_iter(batch_size, features, labels):\n",
    "    num_examples = len(features)\n",
    "    indices = list(range(num_examples))\n",
    "    random.shuffle(indices)  # random read 10 samples 使用shuffle函数打乱数据顺序\n",
    "    for i in range(0, num_examples, batch_size):\n",
    "        j = torch.LongTensor(indices[i: min(i + batch_size, num_examples)]) # the last time may be not enough for a whole batch\n",
    "        yield  features.index_select(0, j), labels.index_select(0, j)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_xc0arq3",
    "id": "1DA3BC30E43E4F76970F712D89BDBC4D",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.1290, -1.1000],\n",
      "        [ 0.4190,  0.7133],\n",
      "        [ 0.2663,  0.7902],\n",
      "        [-1.4476,  0.7135],\n",
      "        [ 1.1073, -0.6617],\n",
      "        [ 0.5837, -0.6599],\n",
      "        [-0.8714,  0.5870],\n",
      "        [ 0.7650,  0.1747],\n",
      "        [-0.8336, -0.1698],\n",
      "        [-1.4624,  0.8141]]) \n",
      " tensor([ 8.2029,  2.5958,  2.0475, -1.1196,  8.6705,  7.5971,  0.4718,  5.1263,\n",
      "         3.1131, -1.4961])\n"
     ]
    }
   ],
   "source": [
    "batch_size = 10\n",
    "\n",
    "for X, y in data_iter(batch_size, features, labels):\n",
    "    print(X, '\\n', y)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_hj6sxxx",
    "id": "1FF819B45B1F44C88012EBB266C10EE8",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 初始化模型参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_g06bzki",
    "id": "6B11AC0E574140CD9C2E722B05D0049D",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.], requires_grad=True)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w = torch.tensor(np.random.normal(0, 0.01, (num_inputs, 1)), dtype=torch.float32)\n",
    "b = torch.zeros(1, dtype=torch.float32)\n",
    "\n",
    "w.requires_grad_(requires_grad=True)\n",
    "b.requires_grad_(requires_grad=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_zvsctyc",
    "id": "A91414B8FDF24835A06B6ADFAEC2C15C",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义模型\n",
    "定义用来训练参数的训练模型：\n",
    "\n",
    "$$\n",
    "\\mathrm{price} = w_{\\mathrm{area}} \\cdot \\mathrm{area} + w_{\\mathrm{age}} \\cdot \\mathrm{age} + b\n",
    "$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_l8xu5kf",
    "id": "8DFF5BDD78884936899E3CE720BEEE3C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def linreg(X, w, b):\n",
    "    return torch.mm(X, w) + b #mm表示矩阵相乘"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false,
    "id": "3F6052C0009048ECAAF06D856C705186",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[1;31mDocstring:\u001b[0m\n",
       "mm(input, mat2, out=None) -> Tensor\n",
       "\n",
       "Performs a matrix multiplication of the matrices :attr:`input` and :attr:`mat2`.\n",
       "\n",
       "If :attr:`input` is a :math:`(n \\times m)` tensor, :attr:`mat2` is a\n",
       ":math:`(m \\times p)` tensor, :attr:`out` will be a :math:`(n \\times p)` tensor.\n",
       "\n",
       ".. note:: This function does not :ref:`broadcast <broadcasting-semantics>`.\n",
       "          For broadcasting matrix products, see :func:`torch.matmul`.\n",
       "\n",
       "Args:\n",
       "    input (Tensor): the first matrix to be multiplied\n",
       "    mat2 (Tensor): the second matrix to be multiplied\n",
       "    out (Tensor, optional): the output tensor.\n",
       "\n",
       "Example::\n",
       "\n",
       "    >>> mat1 = torch.randn(2, 3)\n",
       "    >>> mat2 = torch.randn(3, 3)\n",
       "    >>> torch.mm(mat1, mat2)\n",
       "    tensor([[ 0.4851,  0.5037, -0.3633],\n",
       "            [-0.0760, -3.6705,  2.4784]])\n",
       "\u001b[1;31mType:\u001b[0m      builtin_function_or_method\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "torch.mm??\n",
    "#查看torch.mm的源码.在和鲸上不能成功运行."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_1sta0nq",
    "id": "C9B747281D1842C682F2AEB1F38B959D",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义损失函数\n",
    "我们使用的是均方误差损失函数：\n",
    "$$\n",
    "l^{(i)}(\\mathbf{w}, b) = \\frac{1}{2} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2,\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_r9p6ncn",
    "id": "58A55DD7B46842578BEA1A8689456B1A",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def squared_loss(y_hat, y): \n",
    "    return (y_hat - y.view(y_hat.size())) ** 2 / 2 \n",
    "    #使用y.view方法将y的形状设置为和y_hat一致,以便于能够相减"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_jm7ie9i",
    "id": "0A98B83A8FFD4E84B6EFE8A894643634",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义优化函数\n",
    "在这里优化函数使用的是小批量随机梯度下降：\n",
    "\n",
    "$$\n",
    "(\\mathbf{w},b) \\leftarrow (\\mathbf{w},b) - \\frac{\\eta}{|\\mathcal{B}|} \\sum_{i \\in \\mathcal{B}} \\partial_{(\\mathbf{w},b)} l^{(i)}(\\mathbf{w},b)\n",
    "$$\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_e41t41x",
    "id": "E9676D1B4F80473B894A4ADA3691D2E0",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def sgd(params, lr, batch_size): \n",
    "    for param in params:\n",
    "        param.data -= lr * param.grad / batch_size \n",
    "        # ues .data to operate param without gradient track \n",
    "        #对参数优化的动作不被附加"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_0nsokgo",
    "id": "B18F2D19AA1140478E2E327ECC97F40F",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 训练\n",
    "当数据集、模型、损失函数和优化函数定义完了之后就可来准备进行模型的训练了。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_ht68g0d",
    "id": "8C7AA862EE5A4AEAB3CB980F15870D06",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss 0.031103\n",
      "epoch 2, loss 0.000112\n",
      "epoch 3, loss 0.000050\n",
      "epoch 4, loss 0.000050\n",
      "epoch 5, loss 0.000050\n"
     ]
    }
   ],
   "source": [
    "# super parameters init\n",
    "lr = 0.03 #设置学习率\n",
    "num_epochs = 5 #设置训练周期\n",
    "\n",
    "net = linreg #使用线性网络(模型)\n",
    "loss = squared_loss  #使用均方误差函数\n",
    "\n",
    "# training\n",
    "for epoch in range(num_epochs):  # training repeats num_epochs times\n",
    "    # in each epoch, all the samples in dataset will be used once\n",
    "    #第一层循环,按训练周期\n",
    "    # X is the feature and y is the label of a batch sample\n",
    "    for X, y in data_iter(batch_size, features, labels):\n",
    "        #第二层循环,按数据的批量\n",
    "        l = loss(net(X, w, b), y).sum()  \n",
    "        # calculate the gradient of batch sample loss \n",
    "        l.backward()  \n",
    "        # using small batch random gradient descent to iter model parameters\n",
    "        sgd([w, b], lr, batch_size)  \n",
    "        # reset parameter gradient\n",
    "        w.grad.data.zero_()\n",
    "        b.grad.data.zero_()\n",
    "    train_l = loss(net(features, w, b), labels)\n",
    "    print('epoch %d, loss %f' % (epoch + 1, train_l.mean().item()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_6t702dg",
    "id": "2E791A3F92EF4CCF91E2096630C0E8D9",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(tensor([[ 1.9996],\n",
       "         [-3.4011]], requires_grad=True),\n",
       " [2, -3.4],\n",
       " tensor([4.2000], requires_grad=True),\n",
       " 4.2)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "w, true_w, b, true_b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_pi6pxp6",
    "id": "7E8D79B69557446883330AB1E8DE07E2",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 线性回归模型使用pytorch的简洁实现\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_sdic11w",
    "id": "D5CCF3AE67794558930978F1815C38B9",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.4.0\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from torch import nn\n",
    "import numpy as np\n",
    "torch.manual_seed(1)\n",
    "\n",
    "print(torch.__version__) #版本号,使用本机配置的环境,与视频中不一样,视频中是1.3.0\n",
    "torch.set_default_tensor_type('torch.FloatTensor')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_07nlorv",
    "id": "34B9AE6FB3D64DFD83E93D5CEF9EEE65",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 生成数据集\n",
    "在这里生成数据集跟从零开始的实现中是完全一样的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_k7z5rd0",
    "id": "83C2DB9468394624BB4934DBF194A353",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "num_inputs = 2\n",
    "num_examples = 1000\n",
    "\n",
    "true_w = [2, -3.4]\n",
    "true_b = 4.2\n",
    "\n",
    "features = torch.tensor(np.random.normal(0, 1, (num_examples, num_inputs)), dtype=torch.float)\n",
    "labels = true_w[0] * features[:, 0] + true_w[1] * features[:, 1] + true_b\n",
    "labels += torch.tensor(np.random.normal(0, 0.01, size=labels.size()), dtype=torch.float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "id": "DE689224B7404F6186F18DADEC7491EE",
    "jupyter": {},
    "scrolled": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([-2.7415e+00,  6.4484e+00,  7.5015e+00,  1.0622e+01,  5.6058e+00,\n",
      "         7.0016e+00,  2.5469e-01,  3.0772e+00,  2.6768e+00,  7.4729e+00,\n",
      "         2.5394e+00,  5.6834e+00,  3.0350e+00,  2.9557e-01,  6.8819e+00,\n",
      "         9.8630e+00,  5.3412e+00, -6.3359e-01,  5.1206e+00,  4.9572e+00,\n",
      "         3.9963e+00,  9.7220e+00, -5.0951e-01,  4.2447e+00, -4.2502e+00,\n",
      "         4.7546e+00,  5.2780e+00, -8.8956e-01,  2.0689e+00,  7.6431e-01,\n",
      "         4.0784e+00,  3.0248e+00,  8.6177e-01,  3.8281e+00,  8.3350e+00,\n",
      "         4.4403e+00,  3.6917e+00,  3.5064e+00,  2.7649e+00,  1.0217e+01,\n",
      "         8.4279e+00,  2.5902e+00,  6.3396e+00,  1.8796e+00,  4.8207e+00,\n",
      "        -1.0181e+00,  5.3022e+00,  8.7944e+00,  7.7782e+00,  8.5616e-01,\n",
      "         3.6401e+00, -6.4594e+00,  5.5580e+00,  7.5307e+00, -3.3760e+00,\n",
      "         8.6850e+00,  1.1298e+01,  1.2539e+00,  6.7631e+00,  4.9536e+00,\n",
      "         8.4108e+00,  2.4283e+00,  4.9057e+00,  5.1341e+00,  7.3259e+00,\n",
      "        -2.3421e+00,  1.9409e+00,  3.7598e-01,  9.6972e+00,  8.6646e+00,\n",
      "         6.5763e-01,  9.6679e+00,  3.1126e+00,  5.6888e+00,  5.3645e-01,\n",
      "         1.3852e+01,  1.5447e+00, -1.7215e+00,  1.9818e+00,  4.8352e+00,\n",
      "         3.7219e+00,  2.8717e+00,  8.3226e+00,  6.4146e+00,  8.3744e+00,\n",
      "         6.6907e+00,  9.3879e+00, -2.1739e-02,  8.1850e+00,  2.2149e+00,\n",
      "         8.2128e+00,  6.8768e+00,  4.7064e+00,  5.0895e+00,  6.4469e+00,\n",
      "         5.5100e+00, -5.2339e-01,  1.0021e+01,  4.0512e+00,  6.9580e+00,\n",
      "         5.4890e+00,  7.2558e+00,  7.8405e-01,  5.2900e+00,  1.1398e+01,\n",
      "         5.5980e+00,  3.5560e+00,  6.3230e-01,  1.7349e+00,  8.4134e-01,\n",
      "         2.0480e-01,  7.6708e+00, -1.7823e+00,  6.5960e-01,  8.4116e+00,\n",
      "         5.3011e+00,  1.2481e+00,  8.9022e+00,  7.7159e+00,  1.4169e+01,\n",
      "        -1.0713e+00,  6.1990e+00,  8.2575e-01,  3.6848e+00,  4.2529e+00,\n",
      "         4.9818e+00,  8.9495e+00,  8.2835e+00,  7.6727e+00,  8.1022e+00,\n",
      "         5.0206e+00,  2.8148e+00,  1.1087e+01,  7.6148e+00,  1.7859e-01,\n",
      "         3.3500e-02,  5.1893e+00,  7.3636e+00,  7.9408e-01, -2.9453e-01,\n",
      "         6.0768e+00,  1.3495e+01, -1.7358e+00,  4.0786e+00,  9.2156e-01,\n",
      "         7.3248e+00,  3.9106e+00, -1.1870e-01,  6.4723e+00,  9.7542e+00,\n",
      "         9.2839e+00, -2.5142e-01,  5.3512e+00,  1.1591e+01,  2.5887e+00,\n",
      "         1.5668e-01, -6.2484e-01,  3.9853e+00,  5.4857e+00,  5.3402e+00,\n",
      "         5.5954e+00,  2.1357e+00,  6.5093e+00,  8.4039e+00,  1.0845e+01,\n",
      "         4.1095e+00,  6.0638e-01,  2.9345e+00,  3.4573e+00,  6.3089e+00,\n",
      "         1.7768e+00,  1.8629e+00, -4.4560e+00,  1.1408e+01,  2.7450e+00,\n",
      "         4.4085e+00,  8.5876e+00,  6.1098e-01,  7.3619e+00,  4.0689e+00,\n",
      "         1.4158e+00,  6.7771e+00, -4.3700e-01,  7.1184e-01,  5.8860e+00,\n",
      "        -1.7896e+00,  4.3392e+00,  3.9374e+00,  7.7090e+00,  8.7094e+00,\n",
      "         1.1492e+00,  9.1272e-02,  8.0974e+00, -1.9595e+00,  9.2466e+00,\n",
      "         5.3336e+00,  2.2618e+00,  3.2875e+00, -1.0533e+00,  7.5168e+00,\n",
      "         4.1364e+00,  3.5840e+00,  7.6342e+00,  6.2172e-01,  5.8373e+00,\n",
      "         9.0257e+00,  4.9511e+00,  5.0465e+00, -1.8500e+00,  1.6749e+00,\n",
      "         1.9675e+00,  6.8572e+00,  3.4246e+00,  7.9570e+00,  3.3622e+00,\n",
      "         4.3275e+00,  5.6112e+00,  3.8739e+00,  8.5418e-02, -5.5315e+00,\n",
      "         2.3876e+00,  4.9147e+00,  1.5968e+00,  1.1010e+01,  7.8579e+00,\n",
      "        -1.1649e+00,  5.5982e+00,  1.2131e+01,  3.0786e+00,  3.5738e+00,\n",
      "        -7.7693e-01, -7.0302e-01,  1.4048e+01,  9.6277e+00,  3.7776e+00,\n",
      "         5.4762e+00,  5.8965e+00,  2.4682e+00,  8.1399e+00,  4.3603e+00,\n",
      "         5.3740e+00,  3.7858e+00,  6.1171e+00,  3.3950e+00, -3.6828e+00,\n",
      "         9.3017e+00,  7.0641e+00,  1.1579e+00, -1.9516e+00, -1.0643e+01,\n",
      "         4.8205e-01,  3.8156e+00,  3.9813e+00,  3.6038e+00,  1.0312e+01,\n",
      "        -1.0843e+00,  4.6277e-01,  5.5238e+00,  2.3785e+00,  7.4756e+00,\n",
      "        -3.1612e+00,  6.8708e+00,  7.6571e+00,  5.2535e+00,  5.2627e+00,\n",
      "         8.6120e+00,  2.1156e+00,  1.3273e+00,  7.4643e+00,  4.6703e+00,\n",
      "         1.2211e+01, -2.1165e+00,  3.1609e+00,  5.1926e+00,  4.9797e+00,\n",
      "         8.2018e-01,  3.6512e+00, -1.6252e+00,  1.4181e+01,  1.6548e+00,\n",
      "         8.2073e+00,  9.9094e+00,  1.3512e+01,  3.5188e+00,  7.1786e+00,\n",
      "         6.5200e+00,  1.1044e+01,  5.0723e+00,  3.5224e+00,  5.4297e+00,\n",
      "         5.8187e+00,  3.5754e+00,  5.7824e+00,  4.0091e+00,  1.1120e+00,\n",
      "         1.1788e-02,  3.0827e+00,  5.5157e+00,  8.7343e+00,  8.1703e+00,\n",
      "         2.7868e+00,  3.1723e+00, -4.3052e+00,  2.3342e+00,  4.8702e+00,\n",
      "         3.6177e+00,  7.2552e+00, -4.4578e-01, -2.8074e+00,  1.5185e+00,\n",
      "        -2.0584e+00,  5.5323e+00, -1.9437e+00,  1.1155e+00,  4.0890e+00,\n",
      "         1.2966e+01,  3.2046e+00,  1.2471e+01,  6.4211e+00,  2.0021e-01,\n",
      "        -1.8465e-01, -1.2758e+00,  9.0561e+00,  1.0205e+01, -1.1987e+00,\n",
      "         1.7615e+00,  4.5977e+00,  8.1568e-01,  4.7991e-01,  4.1256e+00,\n",
      "         4.1603e+00,  3.3437e+00,  7.6919e+00, -1.6235e+00, -7.2391e-02,\n",
      "         3.8628e+00,  2.4069e+00,  3.6606e+00,  1.1438e+01,  5.4285e+00,\n",
      "         4.2202e+00,  5.7906e+00,  5.7340e+00,  1.2846e+00,  7.9317e-01,\n",
      "        -8.7281e+00,  5.9058e+00,  6.0615e-01,  6.9929e+00,  3.9536e-01,\n",
      "         6.8120e+00, -8.0277e-01,  3.6334e+00,  6.1505e+00, -2.4504e+00,\n",
      "         4.8358e+00, -1.4017e+00,  1.6276e+00,  1.1189e+01,  8.6970e+00,\n",
      "         7.9091e-01,  4.6504e+00,  2.0242e+00,  2.8283e-01,  3.3727e+00,\n",
      "        -9.3069e-01,  2.4417e+00,  7.2004e+00,  7.4991e+00,  4.5572e+00,\n",
      "         1.3269e+01,  1.2396e+00,  6.2156e+00,  2.1986e+00,  1.2311e+00,\n",
      "         6.5275e-01,  4.5634e+00, -3.5082e+00, -4.8780e-01,  2.8130e-01,\n",
      "         1.9894e+00,  9.0417e+00,  7.4165e+00,  1.2953e+00, -7.7495e+00,\n",
      "        -2.4843e-01,  7.2721e+00, -6.3772e-01,  1.0277e+01,  1.3370e+01,\n",
      "         1.2486e+00,  8.6123e+00,  7.3294e+00,  5.9028e+00,  1.4426e+01,\n",
      "         6.5526e+00, -2.9908e+00,  3.5551e+00,  4.3778e+00,  6.0484e+00,\n",
      "         8.1554e+00, -1.0440e+00, -2.6272e-01,  8.6392e+00, -3.1047e+00,\n",
      "         3.5262e+00,  2.7895e+00,  6.0958e+00,  4.9808e+00,  3.3165e+00,\n",
      "        -6.6615e+00,  4.1916e+00,  4.9241e+00, -2.3225e+00,  7.1083e+00,\n",
      "         6.0847e+00,  8.1969e+00,  3.0483e-01,  3.0981e+00,  1.1027e+01,\n",
      "         1.0334e+01,  6.2772e+00,  4.0363e+00,  7.0953e+00,  7.7673e+00,\n",
      "         4.1666e+00,  6.0728e+00,  1.9047e+00,  6.0713e+00,  1.1982e+01,\n",
      "        -1.2925e+00,  1.1063e+00,  7.7154e+00,  2.9341e+00,  1.0246e+00,\n",
      "         7.6856e+00,  1.9114e+00,  2.9007e+00,  2.8011e+00, -7.0707e+00,\n",
      "         2.0734e+00,  1.7165e+00,  6.5038e+00,  9.4805e-01,  1.2594e+01,\n",
      "         7.0307e+00,  3.3376e+00,  3.4735e+00,  7.0897e+00,  4.6467e+00,\n",
      "         1.0094e+00,  3.4586e+00,  2.6871e+00, -3.0385e+00,  5.5739e+00,\n",
      "         5.8942e+00,  9.6294e-01,  8.9839e+00, -1.6180e+00, -1.7349e+00,\n",
      "         5.0950e+00,  3.4450e+00,  2.2846e+00,  3.3896e+00,  9.0813e+00,\n",
      "         6.1446e+00,  3.4630e+00,  6.7765e+00,  4.0531e+00,  2.2116e+00,\n",
      "         9.3686e+00,  5.5084e+00,  8.7979e-01,  5.3595e+00, -9.6417e-01,\n",
      "         1.2209e+00,  5.9774e+00,  3.1150e+00,  8.1626e+00, -1.2544e+00,\n",
      "         3.4739e+00, -8.2647e-02,  8.1526e-01,  3.7412e+00, -6.6444e+00,\n",
      "         7.8946e+00,  9.4278e+00,  1.5234e+00,  5.8527e+00,  3.1319e+00,\n",
      "         7.9724e+00,  1.3452e+01,  7.2676e+00, -2.1760e+00,  5.7914e+00,\n",
      "         9.8879e-01,  2.9702e+00,  1.0954e+01,  5.5669e+00,  4.8875e+00,\n",
      "         7.2885e+00,  3.1255e+00,  2.3761e+00,  4.0256e+00,  8.0292e+00,\n",
      "         3.7789e+00, -3.1155e+00,  5.4046e+00,  3.0498e+00,  7.8279e+00,\n",
      "         1.4162e+01,  7.3898e+00,  6.2750e-01,  2.8558e+00,  1.0370e+00,\n",
      "         1.0170e+01,  7.1611e-01,  3.2805e+00,  6.4312e+00,  7.9392e+00,\n",
      "         2.9227e+00, -1.3594e+00,  8.9104e+00,  7.6340e+00,  3.1865e+00,\n",
      "         5.8194e+00,  1.5354e+00, -2.3481e+00,  4.0221e+00,  4.3938e-01,\n",
      "         1.2454e+01,  6.5483e-01,  3.7880e+00,  1.9424e+00,  3.5993e+00,\n",
      "         4.2208e+00,  6.3425e+00,  1.3978e+00, -1.2752e+00,  7.4509e+00,\n",
      "         7.4403e+00,  5.2611e+00,  7.1956e+00,  8.9420e+00,  2.8508e+00,\n",
      "         3.0545e+00,  2.2735e+00,  2.7266e+00,  4.3915e+00,  7.6355e+00,\n",
      "         4.9255e+00,  1.6443e+01,  1.0587e+01, -3.8434e+00,  5.7537e+00,\n",
      "        -1.7573e+00,  3.4483e+00,  1.4078e+01,  1.0772e+01,  4.3537e+00,\n",
      "         2.8718e+00,  6.8905e+00,  7.3178e+00,  7.8870e+00,  6.2671e+00,\n",
      "         7.0715e+00,  9.6353e-01,  4.0752e+00,  3.2818e+00,  5.5610e+00,\n",
      "         2.5077e+00,  6.5959e+00,  1.3075e+00,  5.2164e+00,  9.9600e-01,\n",
      "         1.4849e+00,  8.3168e+00,  5.9953e+00,  7.0480e+00, -3.2302e+00,\n",
      "         8.5582e+00,  7.2338e+00,  6.6527e+00,  8.2729e+00,  3.0282e+00,\n",
      "         3.6816e+00,  7.3777e-01, -1.9303e+00, -1.0980e+00, -9.8412e-01,\n",
      "         5.5960e+00,  6.6981e+00,  6.9398e+00,  5.0367e+00,  3.8262e+00,\n",
      "         5.1143e-01,  1.7819e+00,  4.8008e+00,  5.8736e-01,  3.9473e+00,\n",
      "         7.0589e+00,  1.0851e+01,  7.1745e+00, -3.3870e+00, -7.2824e+00,\n",
      "         6.0513e+00,  5.7691e+00,  9.0627e+00,  1.0008e+01,  1.3318e+01,\n",
      "         8.6311e+00,  7.9596e+00,  4.4508e+00,  7.6940e-01,  8.2925e+00,\n",
      "         6.4217e+00, -2.2854e+00, -1.4828e+00,  7.9607e+00, -1.0090e+00,\n",
      "         5.7984e-01,  4.5875e+00,  8.9850e+00,  4.2959e-01,  3.5058e+00,\n",
      "         4.9752e+00,  7.4959e+00,  4.9540e+00,  4.7602e+00, -1.0270e+00,\n",
      "         8.8919e+00,  1.3162e+01,  2.8545e+00, -1.1783e+00,  8.8888e+00,\n",
      "         2.9658e-02,  2.6299e+00,  4.1440e+00,  5.5575e+00,  2.7457e+00,\n",
      "         2.9568e+00,  6.1843e+00,  4.6062e+00,  1.3208e+01,  7.2888e+00,\n",
      "         1.1558e+00,  2.1109e+00,  2.0684e+00, -1.8537e+00,  3.6684e+00,\n",
      "         1.9723e+00,  6.9183e+00,  5.9034e+00,  1.7179e+00,  1.3428e+01,\n",
      "         6.1087e+00,  8.6129e+00,  1.7953e+00,  9.5475e+00, -2.1751e+00,\n",
      "         4.3744e+00,  2.0815e+00,  1.7726e+00,  2.7986e+00,  1.7565e+00,\n",
      "         1.8367e+00, -2.1958e+00,  6.7676e+00,  4.0547e+00,  4.7732e+00,\n",
      "         9.0255e+00,  5.0551e+00, -5.6439e-01,  4.9794e+00,  2.0924e+00,\n",
      "         4.0819e+00,  4.6069e+00,  7.9830e+00,  5.2313e+00,  2.7210e-01,\n",
      "         6.5903e-02,  3.1025e+00,  3.6537e+00,  6.0999e+00,  7.1989e+00,\n",
      "         7.3577e+00,  6.4286e+00,  2.2060e-01,  6.7899e+00,  4.0862e+00,\n",
      "         8.9408e+00, -3.0594e+00,  6.2901e+00,  9.8696e+00,  9.8361e+00,\n",
      "         6.3082e+00,  2.1248e+00,  7.0515e+00,  1.7624e+00,  4.2329e+00,\n",
      "         6.8468e+00,  5.2929e+00,  1.0366e+01,  4.3662e+00,  3.5575e+00,\n",
      "         1.7031e+00,  1.4941e+01,  5.4061e+00,  2.3815e+00,  5.1572e+00,\n",
      "         5.4466e+00,  2.9847e+00,  7.4086e+00,  1.7590e+00,  2.5419e+00,\n",
      "        -7.6898e-03,  4.6591e+00,  1.5568e+00,  1.9435e+00,  9.3204e-01,\n",
      "         5.0600e+00,  5.7328e+00, -1.8670e+00,  1.9608e-01,  2.2105e+00,\n",
      "         4.6758e+00,  2.2047e+00,  5.6547e+00,  6.2709e+00,  2.8564e+00,\n",
      "         1.2265e+00, -4.3862e-01,  6.3263e+00,  3.7893e+00,  4.7690e+00,\n",
      "         1.4016e+00,  2.7300e+00,  9.4225e+00, -2.0650e+00,  4.6251e+00,\n",
      "        -2.2869e+00,  3.8371e+00,  3.8919e+00,  6.3836e+00,  8.4491e+00,\n",
      "         5.0820e+00,  5.7859e+00,  5.6542e+00,  5.6330e+00,  4.2925e+00,\n",
      "         9.0739e+00, -8.7846e-01, -3.4918e+00,  6.6630e+00,  4.4728e+00,\n",
      "        -1.7074e+00,  8.0661e-01, -2.6502e+00,  5.0893e+00,  3.9071e+00,\n",
      "        -6.7852e-01,  4.4685e+00,  4.3433e+00, -7.1225e-01,  1.1153e+00,\n",
      "         4.7177e+00,  2.2346e+00,  2.4714e+00,  1.8670e+00, -5.0354e+00,\n",
      "        -9.8611e-01,  9.4039e+00, -3.9345e+00,  1.0749e+00,  7.3345e+00,\n",
      "         6.0713e+00,  6.0912e+00,  9.6836e+00, -1.3105e+00,  1.5610e+00,\n",
      "         5.2602e+00,  6.8823e+00,  7.6123e+00,  6.9076e+00,  4.7348e+00,\n",
      "         3.0205e+00,  5.4539e+00,  5.5787e+00,  5.2597e+00,  5.0332e+00,\n",
      "         6.5131e+00,  1.8712e+00, -2.0293e+00,  4.1476e+00,  3.9016e+00,\n",
      "         6.0642e+00,  1.4200e+00,  2.3395e+00,  2.7615e+00, -4.7446e-03,\n",
      "         6.2843e-01, -1.4526e+00, -2.0570e+00, -1.8781e+00,  1.7613e+00,\n",
      "        -4.8267e+00,  9.0011e+00, -5.2448e-01,  9.0598e+00, -4.2494e+00,\n",
      "        -1.0169e+00, -3.3111e-01,  5.1935e+00,  4.4285e+00,  6.4701e+00,\n",
      "        -1.2182e+00,  5.1205e+00, -1.4588e+00,  1.1944e+00,  4.5327e+00,\n",
      "         3.7835e+00, -9.7711e-01,  6.8019e+00,  6.7202e+00,  4.3634e+00,\n",
      "         5.3386e+00,  8.1469e+00,  9.4052e+00,  5.8003e+00,  6.5106e+00,\n",
      "        -2.7913e+00,  9.6201e+00,  9.6540e+00,  9.3197e+00,  1.5566e+01,\n",
      "         5.4477e+00,  1.1213e+01,  2.7016e+00,  6.8568e+00, -7.7768e+00,\n",
      "         4.0160e+00,  5.7793e+00,  3.6705e+00,  8.6557e+00,  2.2832e+00,\n",
      "         1.2495e+00,  5.9250e+00,  8.6475e+00,  7.5727e+00, -5.1401e+00,\n",
      "         2.6033e+00,  1.0322e+01,  8.6733e+00,  7.9827e+00,  8.4473e+00,\n",
      "         6.4134e+00,  2.6343e+00,  1.6833e+00,  4.6390e+00,  6.9866e+00,\n",
      "         9.3150e-01,  1.2098e+01,  1.4172e+00,  9.0861e+00,  4.1629e+00,\n",
      "         2.6649e+00,  7.7154e-01, -1.6156e-01,  3.3784e+00,  5.5734e+00,\n",
      "         7.0300e+00, -6.1345e+00,  1.2765e+00,  1.2069e+01,  9.6855e+00,\n",
      "         1.3362e+00,  2.1105e+00,  7.1263e+00,  7.7538e-01,  8.6831e+00,\n",
      "         8.1741e+00,  2.4323e+00,  8.3973e+00,  2.2686e+00,  8.0503e+00,\n",
      "         4.0158e+00,  4.3283e+00,  7.9784e+00,  4.6530e+00,  5.9690e+00,\n",
      "         6.9788e+00,  1.8940e+00,  5.0483e+00,  3.8734e+00,  4.0382e+00,\n",
      "         5.2602e+00,  3.9980e+00,  5.9144e+00,  1.0305e+01,  5.9958e+00,\n",
      "         2.2956e+00,  3.8245e+00,  9.2873e-01,  7.1932e+00,  9.5850e+00,\n",
      "         2.7776e+00, -1.5782e+00,  5.8963e+00, -7.2734e-01,  9.7781e-01,\n",
      "         7.9675e+00,  5.9659e+00,  5.8464e+00,  3.9723e+00,  5.8038e+00,\n",
      "         6.4347e+00,  9.8240e+00,  8.8209e+00,  5.4759e+00,  2.5676e+00,\n",
      "         6.5958e+00,  1.1943e+01,  8.2727e-01, -5.1576e+00,  3.4441e+00,\n",
      "         9.4873e-01,  3.2676e+00,  5.9035e+00,  8.2536e+00,  1.9859e+00,\n",
      "         2.8022e+00,  2.7780e+00,  9.4368e+00,  8.7367e+00, -3.7942e+00,\n",
      "         1.0002e+01,  1.2194e+01,  4.4015e+00, -5.4384e+00,  5.0005e+00,\n",
      "         2.2632e+00,  9.8211e+00,  8.5609e+00,  5.3977e+00,  5.2209e+00,\n",
      "         5.1511e+00,  3.1241e+00,  6.6498e-03,  4.4914e+00,  3.8724e+00,\n",
      "         9.1451e+00,  3.9939e-01, -2.5104e+00,  9.3856e+00,  3.8125e+00,\n",
      "         3.4225e+00,  7.8718e+00,  4.4758e+00,  5.1193e+00,  2.0999e+00,\n",
      "         4.0789e+00,  1.5663e+00,  3.6321e+00, -4.5431e-01,  3.7694e+00,\n",
      "         7.8324e+00,  4.4584e+00,  1.7463e+00,  3.1106e+00,  7.5527e+00,\n",
      "        -4.2173e+00,  9.3785e+00,  5.9215e+00,  1.0679e+00,  1.0144e+01,\n",
      "         8.0316e-01,  1.3772e+00,  6.8506e+00,  1.9157e+00,  9.5365e+00,\n",
      "        -3.9700e+00,  5.7991e+00,  5.0738e+00,  5.3933e+00,  3.8162e+00,\n",
      "         4.7538e+00,  4.6826e+00,  5.0640e+00,  1.0040e+00,  7.5766e+00,\n",
      "         2.9169e+00,  1.1551e+01, -3.9024e+00,  2.8942e-01,  5.0105e+00,\n",
      "         6.0380e+00,  1.1977e+01, -3.3481e+00,  6.1478e+00,  5.9471e+00])\n"
     ]
    }
   ],
   "source": [
    "print(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_io6yz0p",
    "id": "0FB74CD3CD784A82B2A422E54BB0DEDD",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 读取数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_bxmqh9f",
    "id": "8704CA375BF04440839AB16AA995E3AB",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import torch.utils.data as Data\n",
    "\n",
    "batch_size = 10 \n",
    "\n",
    "# combine featues and labels of dataset\n",
    "dataset = Data.TensorDataset(features, labels)\n",
    "\n",
    "# put dataset into DataLoader\n",
    "data_iter = Data.DataLoader(\n",
    "    dataset=dataset,            # torch TensorDataset format\n",
    "    batch_size=batch_size,      # mini batch size\n",
    "    shuffle=True,               # whether shuffle the data or not\n",
    "    num_workers=2,              # read data in multithreading\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_nnjw15x",
    "id": "C1FFC0FD8F5741D78AFD26B883BE192C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-0.4394, -0.2733],\n",
      "        [ 0.5088,  0.6244],\n",
      "        [-3.1206,  2.5311],\n",
      "        [-1.1006, -0.1709],\n",
      "        [-1.3650,  0.6217],\n",
      "        [-0.7348, -1.3356],\n",
      "        [-0.8313, -0.2667],\n",
      "        [-0.6025,  0.0201],\n",
      "        [-0.2095,  0.1451],\n",
      "        [ 1.4455, -0.1782]]) \n",
      " tensor([  4.2529,   3.1025, -10.6429,   2.5887,  -0.6377,   7.2721,   3.4450,\n",
      "          2.9341,   3.2818,   7.7090])\n"
     ]
    }
   ],
   "source": [
    "for X, y in data_iter:\n",
    "    print(X, '\\n', y)\n",
    "    break"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_zobpfwu",
    "id": "F9085AAAB3BB45E289329A5EA5446848",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_gxy6vho",
    "id": "28DD8C6981314D148B5FD1915639151C",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearNet(\n",
      "  (linear): Linear(in_features=2, out_features=1, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "class LinearNet(nn.Module):\n",
    "    def __init__(self, n_feature):\n",
    "        super(LinearNet, self).__init__()      # call father function to init \n",
    "        self.linear = nn.Linear(n_feature, 1)  # function prototype: `torch.nn.Linear(in_features, out_features, bias=True)`\n",
    "\n",
    "    def forward(self, x):\n",
    "        y = self.linear(x)\n",
    "        return y\n",
    "    \n",
    "net = LinearNet(num_inputs)\n",
    "print(net)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_q5pjt1j",
    "id": "56CADFC7B65448BC989411C2C9950816",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sequential(\n",
      "  (linear): Linear(in_features=2, out_features=1, bias=True)\n",
      ")\n",
      "Linear(in_features=2, out_features=1, bias=True)\n"
     ]
    }
   ],
   "source": [
    "# ways to init a multilayer network\n",
    "# method one\n",
    "net = nn.Sequential(\n",
    "    nn.Linear(num_inputs, 1)\n",
    "    # other layers can be added here\n",
    "    )\n",
    "\n",
    "# method two\n",
    "net = nn.Sequential()\n",
    "net.add_module('linear', nn.Linear(num_inputs, 1))\n",
    "# net.add_module ......\n",
    "\n",
    "# method three\n",
    "from collections import OrderedDict\n",
    "net = nn.Sequential(OrderedDict([\n",
    "          ('linear', nn.Linear(num_inputs, 1))\n",
    "          # ......\n",
    "        ]))\n",
    "\n",
    "print(net)\n",
    "print(net[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_fl434p3",
    "id": "1BE602743BCD4C5D948A24212760162D",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 初始化模型参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_zdl7vmt",
    "id": "025B064D1ED1432385DEE75240A790F6",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Parameter containing:\n",
       "tensor([0.], requires_grad=True)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch.nn import init\n",
    "\n",
    "init.normal_(net[0].weight, mean=0.0, std=0.01) #innit模块中的normal_初始化方法\n",
    "init.constant_(net[0].bias, val=0.0)  \n",
    "# or you can use `net[0].bias.data.fill_(0)` to modify it directly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_7s9m78k",
    "id": "C6A909A717B545E6802264EBD711588D",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter containing:\n",
      "tensor([[-0.0142, -0.0161]], requires_grad=True)\n",
      "Parameter containing:\n",
      "tensor([0.], requires_grad=True)\n"
     ]
    }
   ],
   "source": [
    "for param in net.parameters():\n",
    "    print(param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_l729glu",
    "id": "BBFF587F757A4C7EB49AD0D536AD363E",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_or1wah4",
    "id": "B721F8DD4811434BB1984B5B2DABC143",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "loss = nn.MSELoss()    \n",
    "# nn built-in squared loss function \n",
    "#直接调用nn模块中的均方误差损失函数\n",
    "# function prototype: `torch.nn.MSELoss(size_average=None, reduce=None, reduction='mean')`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_zyt512e",
    "id": "6490FA20F3D4462CB2B98902F694E525",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 定义优化函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_pmx4gbq",
    "id": "1998CEB53B534F178AC6223011627B0B",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SGD (\n",
      "Parameter Group 0\n",
      "    dampening: 0\n",
      "    lr: 0.03\n",
      "    momentum: 0\n",
      "    nesterov: False\n",
      "    weight_decay: 0\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.03)   \n",
    "# built-in random gradient descent function \n",
    "#调用随机梯度下降函数作为优化函数\n",
    "print(optimizer)  \n",
    "# function prototype: `torch.optim.SGD(params, lr=, momentum=0, dampening=0, weight_decay=0, nesterov=False)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_n2klgfl",
    "id": "090AC5BD4E214B75BD7C4AB9B68720D0",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "### 训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_qj2fl3l",
    "id": "A4B0F83F71F94728811A619F1AE74CD2",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 1, loss: 0.000226\n",
      "epoch 2, loss: 0.000171\n",
      "epoch 3, loss: 0.000046\n",
      "epoch 4, loss: 0.000059\n",
      "epoch 5, loss: 0.000123\n",
      "epoch 6, loss: 0.000114\n",
      "epoch 7, loss: 0.000039\n",
      "epoch 8, loss: 0.000044\n",
      "epoch 9, loss: 0.000068\n",
      "epoch 10, loss: 0.000172\n",
      "epoch 11, loss: 0.000100\n",
      "epoch 12, loss: 0.000111\n",
      "epoch 13, loss: 0.000123\n",
      "epoch 14, loss: 0.000165\n",
      "epoch 15, loss: 0.000085\n",
      "epoch 16, loss: 0.000139\n",
      "epoch 17, loss: 0.000081\n",
      "epoch 18, loss: 0.000074\n"
     ]
    }
   ],
   "source": [
    "num_epochs = 18 #设置为较大的周期,发现随着训练周期变大,损失会震荡\n",
    "for epoch in range(1, num_epochs + 1): #第一层循环,也是训练周期的循环\n",
    "    for X, y in data_iter:             #第二层循环是对数据循环\n",
    "        output = net(X)\n",
    "        l = loss(output, y.view(-1, 1))\n",
    "        optimizer.zero_grad() # reset gradient, equal to net.zero_grad() \n",
    "        #梯度清零,防止梯度累加\n",
    "        l.backward()\n",
    "        optimizer.step()\n",
    "    print('epoch %d, loss: %f' % (epoch, l.item()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false,
    "graffitiCellId": "id_ke4hsr4",
    "id": "704087439A114181B3A7FE79539127AB",
    "jupyter": {},
    "scrolled": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2, -3.4] tensor([[ 2.0003, -3.3994]])\n",
      "4.2 tensor([4.1999])\n"
     ]
    }
   ],
   "source": [
    "# result comparision #比较真实的结果和偏差以及训练出来的结果和偏差\n",
    "dense = net[0]\n",
    "print(true_w, dense.weight.data)\n",
    "print(true_b, dense.bias.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "graffitiCellId": "id_v7cg0i4",
    "id": "A968DC29635C4CDF8394A6F779661DC5",
    "jupyter": {},
    "mdEditEnable": false,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 两种实现方式的比较\n",
    "1. 从零开始的实现（推荐用来学习）\n",
    "\n",
    "   能够更好的理解模型和神经网络底层的工作原理\n",
    "   \n",
    "\n",
    "2. 使用pytorch的简洁实现\n",
    "\n",
    "   能够更加快速地完成模型的设计与实现\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "499AC6299BB347798995FB804DBF178E",
    "jupyter": {},
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
